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ABSTRACT 

The  Gray-Level  Co-occurrence  Matrix  (GLCM)  based  Trackabiiity  Metric  has  bee  used  at  AMCOM  for  many 
years  for  autotracker  performance  evaluation.  The  origins  of  the  metric  stem  from  the  frustrating  experience  of  trying  to 
use  such  measures  as  delta  T and  signal  to  noise  ratios  to  specify  autotracker  performance  requirements.  This  paper 
presents  the  most  recent  developments  and  improvements  to  the  GLCM  based  Trackabiiity  Metric  (TM)  and  the  resulting 
performance  enhancements.  One  of  the  new  developments  is  the  inclusion  of  a hot  spot  or  most  predominant  feature  metric. 
Additionally,  the  results  of  a sensitivity  study  for  the  accuracy  of  target  segmentation  verses  the  GLCM  TM  performance  is 
presented.  These  new  developments  for  the  Trackabiiity  Metric  should  provide  better  state-of-the-art  performance  prediction 
and  more  accurate  performance  modeling  for  specific  imaging  autotracker  designs.  The  results  of  these  studies  and  the  final 
implementation  of  the  Trackabiiity  Metric  are  presented. 

Keywords:  imaging,  feature,  resolved,  target,  signature,  tracking,  clutter,  metric,  autotracker 


1.  INTRODUCTION 

The  GLCM  Trackabiiity  Metric  (TM)  was  originally  developed  by  the  U.S.  Army  Aviation  and  Missile  Research, 
Development,  and  Engineering  Center  (AMRDEC)  under  the  Advanced  Imaging  Autotrackers  Technology  Base  Program 
to  predict  the  performance  of  autotrackers  for  imaging  infrared  (IR)  missile  seekers.  The  original  application  was  to  predict 
how  well  an  autotracker  could  successfully  segment  and  track  a target  in  its  entirety  from  the  background.  For  some 
applications  full  segmentation  is  important,  for  other  applications  it  is  not.  We  found  that  when  the  TM,  as  originally 
designed,  was  used  for  applications  without  a full  segmentation  requirement,  the  TM  often  did  not  correctly  predict  the 
resulting  autotracker  performance.  Some  targets  with  very  low  full  segmentation  TM  values  were  easily  tracked.  This  was 
sometimes  caused  by  a single  small  predominant  feature  on  the  target,  which  allowed  the  autotracker  to  track  the  single 
feature  itself  against  the  combination  of  the  remainder  of  the  target  and  the  background. 

Several  variations  of  the  TM  were  evaluated  to  include  the  effects  of  a predominant  target  feature  on  autotracker 
performance.  Several  feature  segmentation  and  predominance  measurement  algorithms,  and  methods  of  including  this  new 
measurement  in  the  overall  TM  representation,  were  studied.  Only  the  resulting  design  is  presented  here  in  this  paper. 

User  feedback  indicates  that  the  time  intensive  ground-truthing  process  required  for  the  application  of  the  TM  has 
lead  to  short  cuts  in  the  target  segmentation  process.  The  obvious  question  then  becomes  how  accurate  does  the  target 
segmentation  have  to  be  and  what  is  the  impact  on  the  accuracy  of  the  TM?  The  results  of  a study  addressing  this  issue  are 
addressed  in  section  6. 


2.  GLCM  Traekability  Metric  Revisited 
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The  GLCM  is  a square  matrix  whose  elements  are  an  estimate  of  the  joint  probability  that  a pair  of  pixels  will  have 
two  specific  grayscale  values.  The  formal  definition  of  the  GLCM,  found  over  a region  of  interest  (ROI)  within  an  image 
and  for  a given  orientation,  is  given  by  Equation  2-1. 

Gab(r0’@)  = ^^«Pr{l(m,n)=a,l(m+recos0,n  + resin9)=b}>  (2-1) 

N ' 

where 

l(m,n)  = the  image  intensity  or  grayscale  value  at  the  m’th  row  and  n’th  column, 

Na  b - the  number  paired  pixels  whose  grayscale  values  are  a and  b respectively, 

N = the  total  number  of  pairs  in  the  ROI, 

re  = the  quantized  radial  displacement  between  the  pixels,  and 

0 = the  polar  angle  or  direction  between  the  pixels. 

The  target  and  background  GLCMs  are  found  using  appropriate  windows  as  the  reference  Region  of  Interests 
(ROI)s  for  evaluation  purposes.  Pixel  membership  is  determined  using  segmented-target  ground-truth  data  for  the  image. 
The  structure  length  of  the  target  for  a specified  direction  is  given  by  locating  the  first  peak  in  the  minimum  absolute 
difference  (MAD)  auto-correlation  surface  of  the  target  along  the  direction  of  displacement.  To  account  for  noise  and 
measurement  uncertainty,  the  GLCMs  are  convolved  with  a kernel  formed  from  the  truncated  noise  distribution  appropriate 
for  the  target  area  of  interest.  The  same  convolution  kernel  is  used  for  both  the  target  and  background  GLCMs  and  is  used 
for  four  directional  evaluations. 

The  measured  TM  value  for  the  target  area  of  interest  is  found  by  comparing  the  GLCM  of  the  target  with  that  of 
the  local  background  using  Equation  2-2.  Equation  2-2  yields  a directional  TM  value  along  a specified  orientation.  The 
final  TM  value  is  the  average  of  four  directional  TM  calculations  evaluated  at  0,  90,  and  ±45  degrees. 


(0  < TM  < l) , (2-2) 


where 

G®  = G‘pb  >e)*  k™Ca,  (0> the  involved  target  GLCM, 

G kkb£  = G gkbg  , ©)*  (t) , the  convolved  background  GLCM, 

G^  - the  GLCM  of  the  target  area  in  the  current  frame, 

G^kbg  = the  GLCM  of  the  local  background  in  the  current  frame, 

Kj11^  = the  truncated  uncertainty  kernel  for  the  current  frame, 

£\ f = the  target's  structure  length  along  the  current  direction, 
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0 


= the  eurreni  direction  of  interest  (one  of  the  four  possible  vaiues),  and 


L = the  number  of  available  grayscales  in  the  image. 


Notice  that  both  the  target  and  background  GLCM's  are  found  using  the  target’s  structure  length.  This 
emphatically  couples  the  background  clutter  characterization  with  the  target's  structure.  Note  also  that  Equation  2-2  is  still 
a directional  quantity.  Depending  on  the  intended  application,  this  can  be  exploited  for  comparing  the  target  signature 
along  different  orientations  or  the  orientation  dependence  can  be  integrated  out  b;  averaging  the  results  over  many 
directions.  Further  detail  on  the  GLCM  Trackability  Metric  fundamentals  can  be  found  in  References  1,  4,  and  5. 


3.  PROBLEM  SCENARIO 

The  first  change  made  to  the  TM  was  the  introduction  of  a predominant  feature  measurement.  This  requirement 
has  been  added  due  to  the  application  of  the  TM  to  new  programs.  One  of  the  original  requirements  was  to  measure  the 
likelihood  of  correctly  segmenting  the  entire  target  from  the  background.  Obviously  this  is  not  a requirement  for  a 
successful  track  for  many  applications.  The  particular  case  that  drove  the  Predominant  Feature  Measurement  is  presented 
here.  Figures  3-1  demonstrates  typical  IR  imagery  for  which  the  GLCM  TM  measurement  for  the  whole  target  is  low  but  a 
relatively  small  but  bright  hot  spot  is  present  at  the  target’s  engine  port.  In  this  case,  the  number  of  pixels  comprising  the 
hot  spot  is  small  compared  to  that  of  the  whole  target;  consequently,  the  hot  spot  makes  a rather  small  contribution  to 
GLCM  TM  of  the  whole  target. 


Figure  3-1.  Low  GLCM  TM  with  Predominant  Target  Feature,  Actual  Data 


4.  DETERMINATION  OF  FEATURE  PREDOMINANCE 

A number  of  methods  were  explored  to  measure  the  level  of  target  feature  predominance  that  is  present  within  an 
IR  image.  This  measure  would  be  combined  with  the  classical  whole-target  GLCM  to  form  a composite  measurement. 
Included  in  these  were  the  features  relative  SNR,  the  area  ratio  of  the  feature  with  the  target,  the  energy  ratio  of  the  feature 
with  the  target,  and  many  more  elaborate  formulations. 

The  selected  implementation  was  to  use  the  GLCM  TM  itself  as  a feature  predominance  measurement.  This  new 
implementation  of  the  GLCM  TM  is  the  same  as  the  classical  form  but  with  the  feature  area  mask  used  instead  of  the 
whole-target,  with  the  local  background  area  is  defined  as  anything  but  the  feature.  The  result  is  a normalized  form  that 
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measures  the  textural  content  and  the  intensity  of  the  feature  relative  to  that  of  its  background.  In  such  cases  as  shown  by 
Figure  4-1  and  Equation  4-1  where  a small  but  bright  hot  spot  at  an  engine  port  is  present,  the  extracted  feature  itself  has  a 
very  different  textural  composition  than  that  its  background.  This  provides  an  excellent  measure  of  an  internal  target 
component  that  can  be  used  for  tracking  and,  by  the  nature  of  Equation  4-1 , is  normalized. 


MPF  Baric-  MPF 

ground  Area.  Area 


Figure  4-1.  Feature  GLCM  Trackability  Metric,  Measure  of  Feature  Predominance 
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where 

MPF  = the  Most  Predominant  Feature  of  the  target, 

G “bPF  = G “tPF  ($PF , e)*  K 3^  (t) , the  convolved  MPF  GLCM, 

Gakb  = G^kb8(f^PF,e)*K^Ab(t),  the  convolved  background  GLCM, 

G ^ F = the  GLCM  of  the  MPF  area  in  the  current  frame, 

G ^kbg  = the  GLCM  of  the  local  background  in  the  current  frame, 

K™cAl  - the  truncated  uncertainty  kernel  for  the  current  frame, 

= the  MPF's  structure  length  along  the  current  direction 
Background  = Every  thing  in  the  local  area  except  the  MPF. 


TM^C^e)  = i 


(0  < TM^  < l),  (4-1) 


\ 


5.  COMPOSITE  TRACKABILITY  METRIC 

A rational  solution  for  a composite  track  metric  (TMCOmp)  is  the  normalized  Root  Sum  Square  (RSS)  value  of  the 
whole-target  track  metric  (TMtgt),  which  is  the  classical  form  shown  by  Equation  2-2,  and  the  feature-only  track  metric 
(TMmpf)  as  is  shown  by  Equation  4-5.  This  formulation  takes  into  account  the  special  problem  scenario  exhibited  in  section 
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3.  When  the  GLCM-based  TM  algorithm  is  invoked  using  only  the  MPF  as  the  target  mask  in  such  cases,  the  feature  area  is 
so  much  unlike  the  surrounding  background  that  a relatively  high  value  of  TMmpf  is  measured.  With  this  type  of  case, 
TMcomp  would  take  into  account  the  fact  that  the  whole  target  would  be  difficult  to  track  utilizing  a multi-feature  based 
tracker.  It  would  also  consider  the  fact  that  a portion  of  the  target  could  be  tracked  but  by  ignoring  the  rest  of  the  target.  The 
normalized  RSS  combines  these  two  measurements  with  the  proper  weighting,  never  allowing  TMcomp  to  be  less  than  TMtgt. 
In  situations  for  which  the  whole-target  structure  very  closely  resembles  the  background  and  no  predominant  feature  exists, 
TMcomp  would  be  measured  as  a relatively  low  value  since  TMmpf  would  essentially  be  zero.  The  TMcomp  formulation  shown 
by  Equations  5-1  and  5-2,  and  the  properties  of  which  shown  by  Figure  5-1  exhibit  the  nature  of  the  composite  metric.  The 
normalized  RSS  seems  to  be  the  appropriate  formulation  for  the  low  TMtgt  /hot  spot  condition  as  compared  to  a non- 
normalized  RSS  formulation,  which  was  tested  during  algorithm  development.  The  non-normalized  RSS,  limited  to  1 does 
not  achieve  the  objective  since  a very  predominant  hot  spot  and  a low  TMtgt  results  in  a saturated  value  with  too  much 
weighting  on  the  feature  component.  More  detail  on  the  implementation  and  test  results  can  be  found  in  reference  6. 


For 

TM  MPF  > TM  tgt , 

TM  comp  - -yjl  / 2(tM  tgt  ‘ + TM  MPF " ) 

(0  < TMcomp  < 1), 

(5-1) 

For 

TMMpf  <=TMtgt, 

TMcomp  =TMtgl 

(0<TMcomp  <1), 

(5-2) 

FIGURE  5-1.  Composite  GLCM  Trackability  Metric  Properties 


6.  GLCM  TM  SENSITIVITY  TO  GROUNDTRUTH 

Exact  ground-truth  is  available  for  synthetic  image  sequences,  but  is  not  available  for  captive  flight  data.  In  order  to 
analyze  tracker  performance  against  captive  flight  data,  the  ground-truth  data  must  be  generated  by  hand.  This  manually 
generated  ground-truth  data  is  obviously  not  as  accurate  as  it  would  be  if  it  had  come  from  a simulation.  The  first  questions 
become,  how  does  human  error  in  the  ground-truth  data  affect  the  TM  and  can  one  ground-truth  an  image  sequence  well 
enough  to  get  reasonable  TM  values?  Once  we  have  these  answers,  we  must  develop  and  provided  a tool  and  technique  for 
ground-truthing  that  reduces  the  workload  enough  to  make  the  TM  a usable  product. 

The  ground-truth  can  have  many  sources  of  inaccuracy.  Some  of  the  sources  noted  in  the  past  have  been  the  inclusion  of 
background  pixels  in  the  target  definition,  the  omission  of  some  of  the  target  pixels  from  the  target  definition,  and  target 
definitions  consisting  of  a rectangular  box  approximation  of  the  target  instead  of  an  accurate  segmentation.  These 
inaccuracies  are  always  a direct  result  of  the  labor  and  time  intensive  Groundtruthing  process.  Shortcuts  are  required  to 
allow  the  TM  to  become  a cost  effective  tool. 


7.0  TRACKLAB  GROUND-TRUTHING  TOOL 
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The  ground-truthing  capabilities  of  TrackLab  were  enhanced  with  the  inclusion  of  three-dimensional  models  of  a 
T-72  main  battle  tank  and  a BMP2  armored  personnel  carrier.  This  greatly  reduces  the  amount  of  time  required  to  ground- 
truth  captive  flight  test  (CFT)  data.  Keyboard  commands  where  added  to  TrackLab  to  control  the  position,  scaling,  and 
rotation  of  the  models.  Figure  7-1  shows  a MICOM  IR  Seeker  Analysis  Tool  (MIRSAT)  seeker  image  with  an  overlay  of 
the  3-D  target  model. 


Figure  7-1.  3-D  Target  Model  Overlay. 


The  position,  scaling,  and  rotation  values  for  the  model  are  stored  as  a keyframe  and  are  interpolated  in  between 
keyframes.  The  user  then  has  the  option  of  initiating  a Minimum  Absolute  Difference  (MAD)  correlation  to  accommodate 
minor  jumps  in  the  imagery.  A new  reference  image  is  captured  every  five  frames,  or  whenever  a keyframe  is  reached. 

With  the  assistance  of  the  TrackLab  tool,  an  experienced  user  can  ground-truth  an  image  sequence  in  a respectable 
amount  of  time.  The  image  sequence  represented  by  the  image  in  Figure  7-1  contained  1150  images  and  was  ground- 
truthed  in  approximately  one  hour.  There  were  81  keyframes  defined  during  the  process.  The  operator  identified  the  key 
frames  when  there  was  visible  error  in  the  target  model  placement  and  the  target  in  the  image.  The  operator,  while  trying 
to  do  a good  job,  was  instructed  to  “hurry”  in  order  to  reduce  the  time  required  for  Groundtruthing  as  much  as  possible. 
The  tradeoff  for  accurate  TM  measurements  is  time  and  money  for  the  ground-truthing  process. 

TM  Groundtruthing  Trade  Study 

This  study  was  undertaken  to  determine  how  sensitive  the  GLCM  TM  is  to  the  accuracy  of  the  ground-truth  data. 
The  first  step  in  this  study  was  to  use  a synthetic  sequence,  which  included  “perfect”  ground-truth  data.  The  rows  of 
ground-truth  data  were  then  shifted  up  by  one,  two,  and  three  rows.  The  GLCM  TM  was  then  run  using  the  original 
ground-truth  as  well  as  the  shifted  versions.  Figure  7-2  shows  the  results. 
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Figure  7-2. 

The  output  displayed  has  been  filtered  to  make  the  chart  more  readable.  Figure  7-3  shows  the  percent  errors  of  the  shifted 
curves  with  respect  to  the  perfect  one.  The  curves  are  just  what  would  be  expected.  At  the  start  of  the  sequence,  when  the 
target  is  small,  minor  errors  have  major  effects.  And  near  the  end  of  the  sequence,  the  target  is  much  larger  taking  up 
several  rows  of  the  image.  Therefore,  minor  errors  in  the  ground-truth  placement  have  little  effect. 


Shift  Error 


Figure  7-3. 
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The  next  step  of  the  study  was  to  use  TrackLab  to  manually  ground-truth  the  synthetic  imagery.  This  was  done  in  a “quick 
and  dirty”  manner  to  give  worst-case  results.  Figure  7-4  shows  the  TM  values  calculated  using  the  perfect  and  manual 
ground-truth  data  and  Figure  7-5  shows  the  percent  error. 


Figure  7-4. 


Manual  vs.  Perfect 


- Manual 


Figure  7-5. 
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An  interesting  point  about  Figure  7-5  is  that  most  of  the  TM  values  generated  from  the  manual  ground-truth  are  within 
10%  of  the  values  from  the  perfect  ground-truth.  In  fact,  the  average  error  over  all  frames  is  2.96%.  Figure  7.6  shows  the 
error  and  frequency  of  each  key  frame  entry. 


With  these  respectable  results  in  hand  we  decided  to  increase  the  number  of  keyframes  to  see  how  accurate  we 
could  get  the  TM.  In  review  it  took  approximately  one  hour  for  the  initial  keyframing  exercise,  resulting  in  81  keyframes. 
By  adding  an  additional  76  keyframes  at  a cost  of  45  minutes  of  operator  time,  the  average  TM  error  over  all  frames  was 
reduced  to  2.45%.  With  the  error  dropping  only  0.5%  the  added  labor  cost  is  hardly  worth  it. 


8.  SUMMARY 

This  paper  describes  an  improvement  of  the  GLCM  TM  that  provides  a more  accurate  prediction  in  the 
performance  of  modern  terminal  homing  missile  systems.  This  improvement  consists  of  a normalized  RSS  composite  metric 
value  that  not  only  measures  the  structural  composition  of  the  whole  target  compared  to  that  of  the  surrounding  background, 
as  before,  but  also  the  most  predominant  feature  on  the  target,  such  as  a hot  spot.  The  composite  metric  takes  into  account 
the  potential  for  a tracker  to  track  a single  feature  but  with  some  level  of  penalization  for  only  tracking  a single  feature  of  a 
multi-featured  target.  Additionally,  the  composite  trackability  metric  can  be  calibrated  for  specific  autotracker  systems  on 
the  test  range  for  Go/No-go  test  decisions. 

The  GLCM  TM  ground-truth  accuracy  study  performed  at  MRDEC  shows  a dramatic  change  in  TM  values  for  a 
two-row  shift  or  more  in  the  ground-truth.  It  further  shows  that  the  difference  between  perfect  ground-truth  and  a manual 
ground-truth  aided  with  the  new  three  dimensional  target  models  in  the  TrackLab  tool  averages  only  three  percent.  This 
small  loss  in  accuracy  should  be  acceptable  in  most  applications  for  the  TM. 

The  TrackLab  tools  and  the  GLCM  TM  executable  code  are  Government  owned  and  available  for  Technology 
Transfer  to  private  industry. 
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